{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Variational Multi-modal Recurrent Graph AutoEncoder\n",
    "In this tuorial, we will go through how to run a Variational Multi-modal Recurrent Graph AutoEncoder (VMR-GAE) model for origin-destination (OD) matrix completion. In particular, we will demonstrate how to train the model and evaluate the completion results."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part I: Training\n",
    "In this part, we will show how to train a VMR-GAE model for OD matrix completion on the NYC taxi dataset. In particular, we adopt some training skills from previous works, including data normalization, Kullback-Leibler loss delay and so on."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Visit `paddlespatial/networks/vmrgae/train.py` for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py:943: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working\n",
      "  collections.MutableMapping.register(ParseResults)\n",
      "/opt/anaconda3/lib/python3.8/site-packages/pkg_resources/_vendor/pyparsing.py:3226: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3, and in 3.9 it will stop working\n",
      "  elif isinstance( exprs, collections.Iterable ):\n"
     ]
    }
   ],
   "source": [
    "import argparse\n",
    "import os\n",
    "import numpy as np\n",
    "import paddle\n",
    "import pgl\n",
    "from model import VmrGAE\n",
    "import utils as utils\n",
    "from utils import MinMaxScaler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The VmrGAE class is built upon PaddlePaddle, which is a deep learning framework."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/lib/python3.8/site-packages/ipykernel/ipkernel.py:287: DeprecationWarning: `should_run_async` will not call `transform_cell` automatically in the future. Please pass the result to `transformed_cell` argument and any exception that happen during thetransform in `preprocessing_exc_tuple` in IPython 7.17 and above.\n",
      "  and should_run_async(code)\n"
     ]
    }
   ],
   "source": [
    "def prep_env(flag='train'):\n",
    "    # type: (str) -> dict\n",
    "    \"\"\"\n",
    "    Desc:\n",
    "        Prepare the environment\n",
    "    Args:\n",
    "        flag: specify the environment, 'train' or 'evaluate'\n",
    "    Returns:\n",
    "        A dict indicating the environment variables\n",
    "    \"\"\"\n",
    "    parser = \\\n",
    "        argparse.ArgumentParser(description='{} [VMR-GAE] on the task of OD Matrix Completion'\n",
    "                                .format(\"Training\" if flag == \"train\" else \"Evaluating\"))\n",
    "    parser.add_argument('--num_nodes', type=int, default=263, help='The number of nodes in the graph')\n",
    "    parser.add_argument('--timelen', type=int, default=3, help='The length of input sequence')\n",
    "    parser.add_argument('--hidden_dim', type=int, default=32, help='The dimensionality of the hidden state')\n",
    "    parser.add_argument('--rnn_layer', type=int, default=2, help='The number of RNN layers')\n",
    "    parser.add_argument('--delay', type=int, default=0, help='delay to apply kld_loss')\n",
    "    parser.add_argument('--clip_max_value', type=int, default=1, help='clip the max value')\n",
    "    parser.add_argument('--align', type=bool, default=True,\n",
    "                        help='Whether or not align the distributions of two modals')\n",
    "    parser.add_argument('--x_feature', type=bool, default=False,\n",
    "                        help='X is a feature matrix (if True) or an identity matrix (otherwise)')\n",
    "    parser.add_argument('--data_path', type=str, default='./data/NYC-taxi', help='Data path')\n",
    "    parser.add_argument('--checkpoints', type=str, default='./nyc/checkpoints', help='Checkpoints path')\n",
    "    parser.add_argument('--device', type=str, default='cpu', help='cpu or gpu')\n",
    "    if flag == \"train\":\n",
    "        parser.add_argument('--iter_num', type=int, default=10, help='The number of iterations')\n",
    "        parser.add_argument('--learning_rate', type=float, default=0.0001, help='delay to apply kld_loss')\n",
    "        parser.add_argument('--result_path', type=str, default='./nyc/results', help='result path')\n",
    "    else:\n",
    "        parser.add_argument('--sample_time', type=int, default=10, help='The sample time for point estimation')\n",
    "\n",
    "    args = parser.parse_known_args()[0]\n",
    "\n",
    "    if flag == \"train\":\n",
    "        if not os.path.exists(args.checkpoints):\n",
    "            os.makedirs(args.checkpoints)\n",
    "        if not os.path.exists(args.result_path):\n",
    "            os.makedirs(args.result_path)\n",
    "    else:\n",
    "        if not os.path.exists(args.checkpoints):\n",
    "            print('Checkpoint does not exist.')\n",
    "            exit()\n",
    "\n",
    "    primary_flow = np.load('%s/train_data.npy' % args.data_path, allow_pickle=True)\n",
    "    supp_flow = np.load('%s/green_data.npy' % args.data_path, allow_pickle=True)\n",
    "    train_data = np.load('%s/train_data.npy' % args.data_path, allow_pickle=True)[-1]\n",
    "    val_data = np.load('%s/val_data.npy' % args.data_path, allow_pickle=True)\n",
    "    test_data = np.load('%s/test_data.npy' % args.data_path, allow_pickle=True)\n",
    "\n",
    "    # scaling data\n",
    "    ground_truths = []\n",
    "    for i in range(len(primary_flow)):\n",
    "        primary_flow[i][0] = np.array(primary_flow[i][0]).astype(\"int\")\n",
    "        primary_flow[i][1] = np.array(primary_flow[i][1]).astype(\"float32\")\n",
    "        ground_truths.append(utils.index_to_adj_np(primary_flow[i][0], primary_flow[i][1], args.num_nodes))\n",
    "    ground_truths = np.stack(ground_truths, axis=0)\n",
    "    if args.clip_max_value == 1:\n",
    "        max_value = 50\n",
    "    else:\n",
    "        print(np.concatenate(primary_flow[:, 1]).max())\n",
    "        max_value = np.concatenate(primary_flow[:, 1]).max()\n",
    "    primary_scale = MinMaxScaler(0, max_value)\n",
    "    for i in range(args.timelen):\n",
    "        primary_flow[i][1] = primary_scale.transform(primary_flow[i][1])\n",
    "\n",
    "    for i in range(len(supp_flow)):\n",
    "        supp_flow[i][0] = np.array(supp_flow[i][0]).astype(\"int\")\n",
    "        supp_flow[i][1] = np.array(supp_flow[i][1]).astype(\"float32\")\n",
    "    supp_scale = MinMaxScaler(0, np.concatenate(supp_flow[:, 1]).max())\n",
    "    for i in range(args.timelen):\n",
    "        supp_flow[i][1] = supp_scale.transform(supp_flow[i][1])\n",
    "\n",
    "    # load into paddle\n",
    "    mask = np.zeros((args.num_nodes, args.num_nodes))\n",
    "    for i in range(args.timelen):\n",
    "        mask[np.where(ground_truths[i] > (2 / max_value))] = 1.0\n",
    "\n",
    "    target_graph = []\n",
    "    for i in range(len(primary_flow)):\n",
    "        target_graph.append(pgl.Graph(edges=primary_flow[i][0],\n",
    "                                      num_nodes=args.num_nodes,\n",
    "                                      edge_feat={'efeat': paddle.to_tensor(primary_flow[i][1])}))\n",
    "    supp_graph = []\n",
    "    for i in range(len(primary_flow)):\n",
    "        supp_graph.append(pgl.Graph(edges=supp_flow[i][0],\n",
    "                                    num_nodes=args.num_nodes,\n",
    "                                    edge_feat={'efeat': paddle.to_tensor(supp_flow[i][1])}))\n",
    "\n",
    "    mask = paddle.to_tensor(mask)\n",
    "    xs = paddle.to_tensor([np.eye(args.num_nodes) for i in range(args.timelen)])\n",
    "    x = paddle.to_tensor([np.eye(args.num_nodes) for i in range(args.timelen)])\n",
    "    ground_truths = paddle.to_tensor(ground_truths, dtype='float32')\n",
    "\n",
    "    res = {\n",
    "        \"args\": args,\n",
    "        \"primary_flow\": primary_flow, \"primary_scale\": primary_scale, \"target_graph\": target_graph, \"x\": x,\n",
    "        \"mask\": mask,\n",
    "        # \"supp_flow\": supp_flow, \"supp_scale\": supp_scale,\n",
    "        \"supp_graph\": supp_graph, \"xs\": xs,\n",
    "        \"ground_truths\": ground_truths,\n",
    "        \"train_data\": train_data, \"val_data\": val_data, \"test_data\": test_data\n",
    "    }\n",
    "    return res"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Environment Preparation\n",
    "Here we use `argparse` method for arguments setting, including model parameters, file paths and arguments for training. Then, we load the data from the given path and scale them with normalization process. Since we use `PaddlePaddle` as backend, we should also transform the data into `paddle.tensor` form. Note that we use the iteration number as 10 only for demonstration and it need a larger number for training (e.g., 10e5)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "min: 0 max: 50\n",
      "min: 0 max: 25.0\n"
     ]
    }
   ],
   "source": [
    "if __name__ == '__main__':\n",
    "    env = prep_env()\n",
    "    if env['args'].device=='gpu':\n",
    "        paddle.set_device('gpu')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the model and settings\n",
    "The class is defined in the `paddlespatial/networks/vmrgae/model.py`. \n",
    "\n",
    "Check it for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "    model = VmrGAE(x_dim=env[\"x\"].shape[-1], d_dim=env[\"xs\"].shape[-1], h_dim=env[\"args\"].hidden_dim,\n",
    "                   num_nodes=env[\"args\"].num_nodes, n_layers=env[\"args\"].rnn_layer,\n",
    "                   eps=1e-10, same_structure=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Before training, read the checkpoints if available"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Found the model file. continue to train ... \n"
     ]
    }
   ],
   "source": [
    "    if not os.path.isfile('%s/model.pdparams' % env[\"args\"].checkpoints):\n",
    "        print(\"Start new train (model).\")\n",
    "        min_loss = np.Inf\n",
    "        epoch = 0\n",
    "    else:\n",
    "        print(\"Found the model file. continue to train ... \")\n",
    "        model.set_state_dict(paddle.load('%s/model.pdparams' % env[\"args\"].checkpoints))\n",
    "        min_loss = paddle.load('%s/minloss.pdtensor' % env[\"args\"].checkpoints)\n",
    "        epoch = np.load('%s/logged_epoch.npy' % env[\"args\"].checkpoints)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "    optimizer = paddle.optimizer.Adam(learning_rate=env[\"args\"].learning_rate, parameters=model.parameters())\n",
    "    if os.path.isfile('%s/opt_state.pdopt' % env[\"args\"].checkpoints):\n",
    "        opt_state = paddle.load('%s/opt_state.pdopt' % env[\"args\"].checkpoints)\n",
    "        optimizer.set_state_dict(opt_state)\n",
    "    patience = np.Inf\n",
    "    best_val_mape = np.Inf\n",
    "    max_iter = 0"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start train\n",
    "We now initialize the Adam optimizer and start the training procedure. The learning rate is set to 0.0001. Here we can activate the early stop mechanism if need. Then we train the model for 10 epochs for demostration purposes. In each epoch, we receive the losses, the critical intermediate variables, and the completed OD matrix. If the loss goes down, we then save the checkpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/opt/anaconda3/lib/python3.8/site-packages/paddle/fluid/dygraph/math_op_patch.py:251: UserWarning: The dtype of left and right variables are not the same, left dtype is paddle.float32, but right dtype is paddle.float64, the right dtype will convert to paddle.float32\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "epoch: 2, Loss goes down, save the model. pis_loss = 44.056442\n",
      "val MAE: 16.036873800104313 RMSE: 17.80933483450297 MAPE: 2.8293304056819606\n",
      "test MAE: 16.345251850053376 RMSE: 21.365916076839955 MAPE: 2.3583845337423264\n",
      "epoch: 3, Loss goes down, save the model. pis_loss = 42.328651\n",
      "val MAE: 15.313282637162642 RMSE: 17.06790515321285 MAPE: 2.70043636217129\n",
      "test MAE: 15.499441642387241 RMSE: 20.93153726360367 MAPE: 2.1718683112669486\n",
      "epoch: 4, Loss goes down, save the model. pis_loss = 41.584705\n",
      "val MAE: 14.28275656266646 RMSE: 15.98162930406986 MAPE: 2.472713580716669\n",
      "test MAE: 15.514242705176859 RMSE: 21.007513429655813 MAPE: 2.2006020150986467\n",
      "epoch: 5, Loss goes down, save the model. pis_loss = 38.909363\n",
      "val MAE: 14.249638609452681 RMSE: 15.946169632340105 MAPE: 2.430129598982639\n",
      "test MAE: 14.807489628885307 RMSE: 20.77644899688127 MAPE: 2.074672180305502\n",
      "epoch: 6, Loss goes down, save the model. pis_loss = 38.290962\n",
      "val MAE: 13.810019891912287 RMSE: 15.620660113793411 MAPE: 2.40105472737538\n",
      "test MAE: 14.521877270118862 RMSE: 20.475718140967572 MAPE: 1.9985023213783424\n",
      "epoch: 7, Loss goes down, save the model. pis_loss = 37.015015\n",
      "val MAE: 13.105354352430863 RMSE: 14.962157973180718 MAPE: 2.234107032865299\n",
      "test MAE: 14.303715285132913 RMSE: 20.496078742096138 MAPE: 1.8992136004315872\n",
      "epoch: 8, Loss goes down, save the model. pis_loss = 35.775887\n",
      "val MAE: 13.46273382360285 RMSE: 15.521437897585836 MAPE: 2.341136819323192\n",
      "test MAE: 14.37569452267067 RMSE: 20.93425192103425 MAPE: 1.947665051238785\n",
      "epoch: 9, Loss goes down, save the model. pis_loss = 33.074432\n",
      "val MAE: 12.169471263885498 RMSE: 14.039524555823323 MAPE: 1.9944540910678519\n",
      "test MAE: 13.143257772221284 RMSE: 19.954495075708937 MAPE: 1.6984883423363328\n"
     ]
    }
   ],
   "source": [
    "    for k in range(epoch, env[\"args\"].iter_num):\n",
    "        kld_loss_tvge, kld_loss_avde, pis_loss, all_h, all_enc_mean, all_prior_mean, all_enc_d_mean, all_dec_t, \\\n",
    "            all_z_in, all_z_out \\\n",
    "            = model(env[\"x\"], env[\"xs\"], env[\"target_graph\"], env[\"supp_graph\"], env[\"mask\"],\n",
    "                    env[\"primary_scale\"], env[\"ground_truths\"])\n",
    "        pred = env[\"primary_scale\"].inverse_transform(all_dec_t[-1].numpy())\n",
    "        val_MAE, val_RMSE, val_MAPE = utils.validate(pred, env[\"val_data\"][0],\n",
    "                                                     env[\"val_data\"][1], flag='val')\n",
    "        test_MAE, test_RMSE, test_MAPE = utils.validate(pred, env[\"test_data\"][0],\n",
    "                                                        env[\"test_data\"][1], flag='test')\n",
    "        if val_MAPE < best_val_mape:\n",
    "            best_val_mape = val_MAPE\n",
    "            max_iter = 0\n",
    "        else:\n",
    "            max_iter += 1\n",
    "            if max_iter >= patience:\n",
    "                print('Early Stop!')\n",
    "                break\n",
    "        if k >= env[\"args\"].delay:\n",
    "            loss = kld_loss_tvge + kld_loss_avde + pis_loss\n",
    "        else:\n",
    "            loss = pis_loss\n",
    "        loss.backward()\n",
    "        optimizer.step()\n",
    "        optimizer.clear_grad()\n",
    "        if k % 10 == 0:\n",
    "            print('epoch: ', k)\n",
    "            print('loss =', loss.mean().item())\n",
    "            print('kld_loss_tvge =', kld_loss_tvge.mean().item())\n",
    "            print('kld_loss_avde =', kld_loss_avde.mean().item())\n",
    "            print('pis_loss =', pis_loss.mean().item())\n",
    "            print('val', \"MAE:\", val_MAE, 'RMSE:', val_RMSE, 'MAPE:', val_MAPE)\n",
    "            print('test', \"MAE:\", test_MAE, 'RMSE:', test_RMSE, 'MAPE:', test_MAPE)\n",
    "\n",
    "        if (loss.mean() < min_loss).item() | (k == env[\"args\"].delay):\n",
    "            print('epoch: %d, Loss goes down, save the model. pis_loss = %f' % (k, pis_loss.mean().item()))\n",
    "            print('val', \"MAE:\", val_MAE, 'RMSE:', val_RMSE, 'MAPE:', val_MAPE)\n",
    "            print('test', \"MAE:\", test_MAE, 'RMSE:', test_RMSE, 'MAPE:', test_MAPE)\n",
    "            min_loss = loss.mean().item()\n",
    "            paddle.save(all_enc_mean, '%s/all_enc_mean.pdtensor' % env[\"args\"].result_path)\n",
    "            paddle.save(all_prior_mean, '%s/all_prior_mean.pdtensor' % env[\"args\"].result_path)\n",
    "            paddle.save(all_enc_d_mean, '%s/all_enc_d_mean.pdtensor' % env[\"args\"].result_path)\n",
    "            paddle.save(all_dec_t, '%s/all_dec_t.pdtensor' % env[\"args\"].result_path)\n",
    "            paddle.save(all_z_in, '%s/all_z_in.pdtensor' % env[\"args\"].result_path)\n",
    "            paddle.save(all_z_out, '%s/all_z_out.pdtensor' % env[\"args\"].result_path)\n",
    "            paddle.save(model.state_dict(), '%s/model.pdparams' % env[\"args\"].checkpoints)\n",
    "            paddle.save(loss.mean(), '%s/minloss.pdtensor' % env[\"args\"].checkpoints)\n",
    "            paddle.save(optimizer.state_dict(), '%s/opt_state.pdopt' % env[\"args\"].checkpoints)\n",
    "            np.save('%s/logged_epoch.npy' % env[\"args\"].checkpoints, k)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The above is about the training steps, you can adjust it as needed."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part II: Result Evalution\n",
    "Below we will introduce how to use the trained model for OD matrix completion and evaluate the results.\n",
    "\n",
    "Visit `paddlespatial/networks/vmrgae/eval.py` for more details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "min: 0 max: 50\n",
      "min: 0 max: 25.0\n"
     ]
    }
   ],
   "source": [
    "from train import prep_env\n",
    "if __name__ == '__main__':\n",
    "    env = prep_env(flag='eval')\n",
    "    if env['args'].device=='gpu':\n",
    "        paddle.set_device('gpu')\n",
    "\n",
    "    # load VMR-GAE and run\n",
    "    model = VmrGAE(x_dim=env[\"x\"].shape[-1], d_dim=env[\"xs\"].shape[-1], h_dim=env[\"args\"].hidden_dim,\n",
    "                   num_nodes=env[\"args\"].num_nodes, n_layers=env[\"args\"].rnn_layer,\n",
    "                   eps=1e-10, same_structure=True)\n",
    "\n",
    "    if not os.path.isfile('%s/model.pdparams' % env[\"args\"].checkpoints):\n",
    "        print('Checkpoint does not exist.')\n",
    "        exit()\n",
    "    else:\n",
    "        model.set_state_dict(paddle.load('%s/model.pdparams' % env[\"args\"].checkpoints))\n",
    "        min_loss = paddle.load('%s/minloss.pdtensor' % env[\"args\"].checkpoints)\n",
    "        epoch = np.load('%s/logged_epoch.npy' % env[\"args\"].checkpoints)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here we use the same preparation function in training process with `eval` flag to hold the model configurations. Then, we load the model and the available checkpoint."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Start Evaluation\n",
    "We perform the trained model for `sample_time` times and report the mean values as the completion results, as well as the standard deviations."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[20.457312 22.099165 22.369968 ... 20.148964 20.399767 19.612144]\n",
      " [19.998463 21.68787  21.64738  ... 19.538559 20.009933 19.260576]\n",
      " [20.739218 22.463572 21.908607 ... 20.383247 20.573793 20.101337]\n",
      " ...\n",
      " [20.076315 21.49416  21.676641 ... 19.469728 19.679989 18.836117]\n",
      " [19.822569 21.379684 20.878109 ... 19.325747 19.726997 19.063314]\n",
      " [19.820883 21.541578 21.235636 ... 19.330482 19.592773 19.105854]]\n"
     ]
    }
   ],
   "source": [
    "    pred = []\n",
    "    for i in range(env[\"args\"].sample_time):\n",
    "        _, _, _, _, _, _, _, all_dec_t, _, _ \\\n",
    "            = model(env[\"x\"], env[\"xs\"], env[\"target_graph\"], env[\"supp_graph\"], env[\"mask\"],\n",
    "                    env[\"primary_scale\"], env[\"ground_truths\"])\n",
    "        pred.append(env[\"primary_scale\"].inverse_transform(all_dec_t[-1].numpy()))\n",
    "    pred = np.stack(pred, axis=0)\n",
    "    pe, std = pred.mean(axis=0), pred.std(axis=0)\n",
    "    pe[np.where(pe < 0.5)] = 0\n",
    "    print(pe)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.15"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}
